fix(prose-tests): keep the walk's closing turn, and name the one live difference - #572
Merged
Merged
Conversation
This was referenced Jul 27, 2026
Merged
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:47
10fb586 to
2883985
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:49
a9d6186 to
c976b82
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:49
2883985 to
bb9b308
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:50
c976b82 to
ae932a1
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:50
bb9b308 to
d00a14a
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:52
ae932a1 to
d62969d
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:52
d00a14a to
c15985f
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:53
d62969d to
4960fba
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:53
c15985f to
5c8eaa3
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:54
4960fba to
9e7d97c
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:54
5c8eaa3 to
b25026e
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:56
9e7d97c to
96d3eac
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:56
b25026e to
72d1fbf
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:57
96d3eac to
e62a792
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:57
72d1fbf to
7f84584
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:58
e62a792 to
fcba6bb
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:58
7f84584 to
d5175ed
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 15:59
fcba6bb to
db544d9
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 15:59
d5175ed to
c992f1d
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 16:00
db544d9 to
3766bab
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 16:00
c992f1d to
c6d27ea
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 16:01
3766bab to
28f9030
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 16:01
c6d27ea to
ddd8f1b
Compare
leeovery
force-pushed
the
test/prose-discovery-epic-arm
branch
from
July 27, 2026 16:02
28f9030 to
de078e5
Compare
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 16:02
ddd8f1b to
07bdc40
Compare
… difference Both spec-entry cases failed on a claim about the handoff block their walks emitted correctly. The block was missing from the record, so the asserter had nothing to judge and rightly refused to pass an unevidenced step. Four walks, two cases, one cause. The cause is not the parser. Running the hook's own extraction over the settled transcript returns every turn including the one it missed, so the file simply did not hold that turn yet when the hook read it. This hook runs inside the stop sequence rather than after it: the agent has stopped thinking, and the runtime is still writing down the last thing it said. The turn is not lost, only not yet on disk — the stop payload carries it as `last_assistant_message`, which is the runtime handing it over at exactly the moment it is unreadable anywhere else. It is appended as the closing turn when it is not already there. Reading the transcript later instead, from the assert command, would also have worked, but it splits the record across two components and leaves the world holding a pointer rather than the walk; the world stays self-contained this way. Appending only when absent is not deduplication. Two identical turns that are genuinely in the transcript both survive, because a step that ran twice is something the asserter has to see. Truncation is reconsidered with it. The caps were never about disk — the logs are throwaway — but about the asserter's prompt, which every recorded action is read into. That justifies trimming a file body no claim ever rests on, and does not justify trimming evidence: the walk log now caps nothing at all, and a command's output gets a ceiling that bounds a runaway without ever reaching real output. Separately, the one genuine difference between a walk and a live session is written down rather than rediscovered each run. An inline directive is substituted when a skill loads live; a walk reads the prose as a file, so the walker takes the fallback the prose supplies for that case. Correct, expected, and not a marker. The asserter is also told what it costs — the primary arm is the one real sessions take and it is never exercised here, so a claim resting on the substituted value is unprovable rather than passed. It is named as the only such exemption: anything else resembling an environment quirk is a finding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
leeovery
force-pushed
the
fix/prose-closing-turn-and-known-difference
branch
from
July 27, 2026 17:05
07bdc40 to
9fdf6ab
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
Both spec-entry cases failed on a claim about a handoff block their walks emitted correctly. The block was missing from the record, so the asserter had nothing to judge and rightly refused an unevidenced step. Four walks, two cases, one cause.
It isn't the parser. Running the hook's own extraction over the settled transcript returns every turn including the missing one — so the file didn't hold that turn yet when the hook read it.
The hook runs inside the stop sequence, not after it. The agent has stopped thinking; the runtime is still writing down the last thing it said.
The fix
The turn isn't lost, just not yet on disk — the stop payload carries it as
last_assistant_message, the runtime handing it over at exactly the moment it's unreadable anywhere else. It's appended as the closing turn when not already present.Reading the transcript later from
assertwould also have worked, but it splits the record across two components and leaves the world holding a pointer rather than the walk. This keeps the world self-contained.Appending-when-absent is not deduplication. Two identical turns genuinely in the transcript both survive — a step that ran twice is something the asserter must see.
Truncation, reconsidered alongside it
The caps were never about disk (the logs are throwaway) but about the asserter's prompt, which every recorded action is read into. That justifies trimming a file body no claim rests on; it doesn't justify trimming evidence.
The one known live difference, written down
An inline
!`command`directive is substituted when a skill loads live. A walk reads prose as a file, so the walker takes the fallback the prose supplies for exactly that case — correct, expected, and no longer reported as a marker.The asserter is also told what it costs: the primary arm is the one real sessions almost always take and it is never exercised here, so a claim resting on the substituted value is unprovable rather than passed. Named as the only such exemption — anything else resembling an environment quirk is a finding, not a licence.
Test plan
🤖 Generated with Claude Code
Stack